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Abstract: Intelligent Tutoring Systems (ITSs) are beneficial for individual students learning 
in several domains, including mathematics where they have been used to support both 
secondary and elementary students. Collaborative learning may be beneficial to include in 
ITSs, particularly for conceptual knowledge. There is little work on collaborative ITSs, and it 
has mostly focused on older students. We aim to extend this work to elementary school 
students, by extending an ITS for fractions so it supports collaborative learning. We also build 
upon our previous work to further investigate the complementary strengths of collaborative 
and individual learning. In our study, 189 elementary school students worked with a 
conceptual or a procedural fractions ITS, and either individually or collaboratively. Students 
in both ITS conditions learned, but there were no differences in learning between individual 
and collaboration. However, the students working collaboratively spent less time on the tutor, 
indicating potential benefits of collaborative learning on efficiency in this setting. 
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Introduction 

Intelligent Tutoring Systems (ITSs) have been very successful in supporting students’ learning as they work 
individually to solve problems (VanLehn, 2011). Although this individual work has been shown to be 
beneficial, there may be additional benefits from students being able to work collaboratively, especially when 
they are acquiring conceptual knowledge (Mullins, Rummel, & Spada, 2011). Previous research has shown that 
integrating collaboration into an ITS environment can be conducive for learning (Walker, Rummel, & 
Koedinger, 2009). But most of this work has been done with older students and has not been extended to 
elementary school students where collaboration is expected to be more challenging. We aim to investigate if the 
benefits seen with older students can apply to elementary school students and how best to utilize collaboration 
and individual learning within an ITS to support students. 

Although ITSs have been shown to be beneficial to students in many domains (Murray, 2003), they 
have been shown to be particularly successful with mathematics (Ritter, Anderson, Koedinger, & Corbett, 
2007). For elementary school students, fractions are challenging (Moss, 2005), but the learning of fractions can 
be successfully supported through the use of an ITS, showing that ITSs can be beneficial for young learners 
(Rau, Aleven, & Rummel, 2012). ITSs are beneficial to students by providing them with cognitive support as 
they solve a problem (VanLehn, 2011). ITSs provide step-by-step guidance for students both through the use of 
immediate feedback on steps and through on demand hints. That is, students will know right away when an error 
occurs and they can decide to request help from the system to figure out how to do any problem-solving step 
correctly. ITSs also track individual students’ knowledge growth, which enables them to implement a cognitive 
mastery approach with individualized problem selection. ITSs often focus on helping students acquire problem¬ 
solving skills, and by integrating collaboration, we may be able to support students in explaining their reasoning, 
as is specified in the United States common core standards (Common Core State Standard Initiatives, 2015). 

Collaboration is often beneficial for learning (Slavin, 1989; Lou, Abrami, & d’Apollonia, 2001) and is 
a way to get students to express their reasoning (Hausmann, Chi, & Roy, 2004). Collaborative problem solving 
may help students develop a deeper conceptual understanding (Teasley, 1995) through mechanisms such as co¬ 
construction and explanation-giving (Hausmann, Chi, & Roy, 2004; Chi & Wylie, 2014). Through these 
mechanisms, collaborative learning may be beneficial as an addition to standard ITS technology, especially for 
conceptual knowledge where students often need to develop a deeper understanding of concepts in the domain 
(Mullins et al., 2011). 

Although most prior research on ITSs has focused on students working individually, there has been 
some work combining ITSs and collaboration with high school students (Baghaei, Mitrovic, & Irwin, 2007; 
Walker et al., 2009; Diziol, Walker, Rummel, & Koedinger, 2010). Walker, Rummel, and Koedinger (2009) 
found that students working with a tutor that had been redesigned to support peer tutoring (i.e., the tutoring 
system provided support to the student in the role of the peer tutor) achieved learning gains at least equivalent to 



those working individually, demonstrating that collaboration can be successfully combined with ITSs. However, 
much of this work has been done with secondary and college students rather than elementary school students. 
For elementary school students, learning collaboratively may be challenging, especially in STEM domains such 
as mathematics (Mercer & Sams, 2006). Elementary school students often do not have fully developed social 
skills, making collaborative activities more challenging. Also, elementary school students may not have 
developed the vocabulary to discuss complex math concepts and relations. Despite these challenges, 
collaboration may still be effective for elementary school students by allowing them to make their thinking 
explicit and to practice their ability to talk about mathematics (Chi & Wylie, 2014). Even outside of ITSs, few 
studies have investigated whether computer supported collaborative learning (CSCL) can have a positive impact 
on learning with elementary school students. The studies that have been conducted in this area have either 
compared the use of a CSCL setting to face-to-face collaborative learning (i.e., not supported by computers) or 
have focused on technology interventions that mix individual and collaborative learning tasks without 
comparing learning collaboratively to learning individually as we propose to do (Chen & Looi, 2013; Lazakidou 
& Retalis, 2010; Tsuei, 2011). This research has shown positive impacts of young children working in small 
groups and with computers that can be extended to the use of ITSs. 

In addition, collaborative and individual learning may have different strengths. For example, individual 
and collaborative learning activities may be better for acquiring different types of knowledge, such as 
conceptual and procedural knowledge (Mullins et al., 2011). Conceptual knowledge is the implicit and explicit 
understanding of the principles in a domain and how they are interrelated (Rittle-Johnson, Siegler, & Alibali, 
2001) and may be better acquired through collaboration (Teasley, 1995). Procedural knowledge is the ability to 
be able to perform the steps and actions in sequence to solve a problem (Rittle-Johnson, Siegler, & Alibali, 
2001) and may be better acquired through individual work with the opportunity of more practice. Understanding 
the relative strengths of individual and collaborative learning, which we focus on in this paper, may ultimately 
help in designing instructional conditions that effectively combine individual and collaborative learning. 

In a previous study with elementary school students (Olsen, Belenky, Aleven, & Rummel, 2014a), we 
did not find differences in learning gains between students working collaboratively or individually for either 
conceptual or procedural knowledge. However there were several weaknesses in the study that we are 
addressing with the current study. In the previous study, the procedural tasks may have been too difficult for the 
students. Also, the total instruction time was only 45 minutes, which may not have been enough for differences 
in learning gains to emerge. The current study took place in a more realistic classroom setting (the previous 
study was a pull-out study), allowing the students to speak more freely and to work with the tutor across 
multiple sessions. The study involved 189 students and ran over five class periods to provide time for multiple 
units to be completed. To test our hypotheses that a collaborative ITS can effectively support young learners, 
and that collaborative and individual learning have different strengths for conceptual and procedural knowledge, 
we used two separate 2-condition between-subjects designs. Each compared a collaborative and an individual 
learning conditions, one with a conceptually-oriented set of tutor problems, and one with a procedurally- 
oriented set of tutor problems. This design allows us to compare between individual and collaborative learning 
within both conceptually-oriented and procedurally-oriented tutors. The design is not a true 2x2 design. The 
conceptual and procedural tutors cannot be directly compared because they support different learning goals. 

Methods 

Before we present the study in more detail, we describe our ITS designed to support fractions learning. 

Tutor Design 

Informed by prior work on the Fractions Tutor (Rau et al., 2012) and our previous collaborative tutors for 
equivalent fractions (Olsen et al., 2014a), we developed a new ITS for a range of fractions units. To compare 
students working collaboratively to students working individually, we built two parallel versions of the tutors 
for use in our study, one for collaborative learning and one for individual learning. The ITS versions were built 
with the Cognitive Tutoring Authoring Tools (CTAT), extended to support collaborative tutors (Olsen et al., 
2014b). For both collaborative and individual learning, we created two sets of tutor units, focused on either 
procedural knowledge or conceptual knowledge. For both sets, the units covered were naming, making, 
equivalent, least common denominator, comparing, adding, and subtracting fractions. For each unit, there were 
eight problems. All of the problems within a unit were of the same type and focused on the same procedure or 
concept. The problems were divided between graphical representations with two problems focusing on circles, 
two problems focusing on rectangles, and four problems focusing on number lines. Although the conceptually- 
oriented and procedurally-oriented tutors covered the same units, the materials were fundamentally 



Comparing Fractions 

a Let's compare fractions 

1 What is the least common denominator of the below fractions? 



3 Take the fractions below the number lines and reorder them from the smallest fraction to the largest fraction. 
Smallest Largest 
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4 
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10 


20 


Figure 1 . An example of an individual procedural problem for comparing fractions. The students go through the 
steps of converting fractions as a general mathematical procedure for solving this type of problem. 


different to match different learning goals. For the procedural problem set, the students went through the steps 
needed to solve the problem. For example, for the comparing fractions unit, students would first find the least 
common denominator for the fractions they were trying to compare (see Figure 1) and then convert all of the 
fractions using this common denominator. Once they had completed this process, they were asked to put the 
fractions in order, from smallest to largest. For the conceptual problem set, the students were asked to fill in 
answers to see a pattern in the fractions, and then to fill in the blank for sentences to generalize the patterns to 
fractions concepts. For example, for the comparing fractions unit, the students were asked to mark if fractions 
were larger or smaller than one another. These fractions consisted of one where the denominators were the 
same, one where the numerators are the same, and one that is mixed (see Figure 2). The students were then 
guided to understand that when numerators are the same, then the fraction with the smaller denominator is 
larger, and when the denominators are the same then the fraction with the larger numerator is larger. These fill 
in the blanks were reworded for every other problem so students could not just memorize the slot that each 
answer should go into. 



Figure 2 . An example of a collaborative conceptual problem for comparing fractions. In panel A, each student is 
responsible for half the answers while in panel B the students share the responsibility for the questions. 


In addition to the cognitive support normally provided by an ITS (step-level guidance for problem 
solving) that is used in both the individual and collaborative tutors, the collaborative tutors are also supported 
with embedded collaborative scripts for each tutor problem to provide social support for students (Kollar, 
Fischer, & Hesse, 2006). The collaborative tutors support synchronous, networked collaboration, in which 
collaborating students sit at their own computer and have a shared (though differentiated) view of the problem 
state and different actions/resources available to them. The students sat next to each other and communicated 
through speech. The embedded scripts supported collaboration through a distribution of responsibility to create 





































































accountability and interdependence (Slavin, 1989). The students were responsible for different parts of the 
problem (see Figure 2). Often in the problem, each student had some action that only they could take for the step 
to be completed. This supported both students in needing to contribute. In Figure 2, each student has a set of the 
compared fractions. The students cannot fill in all three of the answers by themselves because they do not have 
access to all of the correct answers. They need to work together with their partner to fill in all of the blanks and 
submit the problem. This distributes the responsibility for each of the steps across both students. In addition, 
some steps created interdependence between the students. An example is for equivalent fractions that if one 
student entered a numerator, this would influence what the correct denominator would be for their partner. The 
embedded scripts provided support for the collaboration to be directly integrated into the ITS. Besides the 
embedded collaboration script, all collaborative and individual tutors were designed to be identical to allow for a 
comparison. During the time with the ITS, students who were collaborating sat next to each other but worked on 
different computers. They communicated through speech, which was recorded. 

Experimental Design and Procedure 

We conducted a study with 189 4 th and 5 th grade students from two schools across two school districts. The 
students came from a total of nine classrooms and five teachers. The experiment took place during the students’ 
regular class periods. All students worked with the fractions ITS described above, either on conceptually- 
oriented tasks only or on procedurally-oriented tasks only, and working either collaboratively all the way 
through or individually all the way through. Thus, we had 4 conditions, which, as discussed above, we analyze 
as two separate 2-conditions designs. Students were assigned to individual and collaborative conditions based on 
class for a quasi-experimental design. This random assignment based on class was done to limit the disruption to 
the class. There were five classes that were assigned to work collaboratively and four classes that were assigned 
to work individually. Within each class, teachers paired their students based on students who would work well 
together and had similar math abilities. These pairs were then randomly assigned to work on the procedurally- 
oriented or the conceptually-oriented problem sets. Within the class there was an even split between students 
working on conceptually-oriented problems and students working on procedurally-oriented problems. 

During the study, if a student’s partner was absent in the collaborative conditions, the student would be 
paired with another student working on their same type of problem set for the remainder of the experiment. If 
there were two students that needed partners who had worked together before, they were paired together. If there 
were an odd number of students who needed a partner, then one student would work individually for the day. 
The teacher informed student pairings with a new partner when there was more than one option. When students 
started with a different partner from the day before, they would begin on the problem set at the place of the 
student that was less far in the problem sets. 

The study ran across five days for each class where each day the students had 45 minutes to work with 
the tutor. On the first day, the students took the pretest individually. When they completed the pretest, they 
moved onto a tutorial that gave some instruction on how to interact with the tutor; otherwise, this was done on 
the second day. The students then worked with the tutor for the next three days in their condition. On the fifth 
day, the students took a posttest individually. 

Test Items 

We assessed students’ knowledge at two different times using two equivalent test forms in counterbalanced 
fashion. The tests targeted both conceptual and procedural knowledge types for all conditions. Each test had 20 
questions, 10 procedural and 10 conceptual where six were isomorphic with the main six fractions units and four 
were near transfer targeting the four upper level fractions units. For each question on the test, the students were 
able to get a point for each step completed correctly. On the tests there were 23 possible conceptual test points 
and 68 procedural test points. Because of the discrepancy in points for the different types of knowledge, for both 
conceptual and procedural test scores, the percentages were used for all analyses. 

Results 

Due to absenteeism during the study, only 146 of the 189 students were used for the analysis. Students were 
excluded if they missed either the pretest or posttest. They were also excluded if they missed more than 1 day of 
working with the tutor. In the collaborative condition, students were excluded if they had more than a total of 
two partners during the tutors. There was no significant difference between conditions with respect to the 
number of students excluded, F( 3, 185) = 0.72, p = .54. There was also no significant difference between 
conditions on the pretest score for either the conceptual test items, F( 3, 142) = 0.49, p = .69, or for the 
procedural test items, F{ 3, 142) = 0.68 ,p = .57. 



As discussed, we used a separate 2-condition between-subjects design to compare an individual and a 
collaborative learning condition for conceptually-oriented and procedurally-oriented tutor problem sets. For the 
analysis, the data was thus treated as two separate data sets within which students working collaboratively or 
individually could be compared. Out of the 146 students used in the analysis, 70 students worked with the 
procedurally-oriented ITS, and 76 students worked with the conceptually-oriented ITS. 


Pre/Posttest Learning Gains 

To investigate whether students learned using our tutors, and if there was a difference in learning between the 
students working collaboratively and students working individually within the two tutor problem sets, we used a 
multilevel approach to take into account the repeated measures of the pretest and posttest and differences 
between teachers. Within this analysis, we treated all students as individuals. We conducted a hierarchical linear 
model (HLM) with student at the first level and teacher at the second level. At level 1, we modeled the pretest 
and posttest scores, and at level 2, we accounted for random differences that could be attributed to the teacher. 
We did not include dyads as a level because of the added complexity of some students working with no partner 
(i.e. individuals), some students having one partner, and some students having two partners because of 
absenteeism. We are aware of non-independence issues such as common fate and reciprocal influence that may 
impact our results (Cress, 2008). We measured the effect size with Pearson’s correlation coefficient (r) where 
0.1 is a small effect size, 0.3 is a medium effect size, and 0.5 is a large effect size. 
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Figure 3 . Learning gains on the conceptually-oriented tutors. Conceptual test items on the left and procedural 
test items on the right. Collaborative students are black and individual students are light gray. 

First, we analyzed data from students who worked with the conceptually-oriented tutors. We tested 
whether there were statistically significant pre/post learning gains and we compared the learning gains between 
students working individually and students working collaboratively. We did so separately (with separate HLMs) 
for gains on the conceptual test items and gains on procedural test items. For the conceptual test items, a 
significant effect between pre and posttests was found with posttest having a higher score, £(74) = -9.29, p < 
.001, r = .73, while no significant effect for individual/collaborative learning, £(70) = -1.25, p = .22, r = .15, nor 
interaction, £(74) = 1.47, p = .15, r = .17, was found (see Figure 3). For the procedural test items, a significant 
effect between pre and posttests was found with posttest having a higher score, £(74) = -6.80, p < .001, r — .62, 
while no significant effect for individual/collaborative learning, £(70) = 0.35, p = .72, r = .04, nor interaction, 
£(74) = 0.39, p = .70, r = .05, was found (see Figure 3). Thus, although there was learning from pretest to 
posttest on both conceptual and procedural test items, there was no difference in learning between students 
working individually and students working collaboratively. To investigate whether there was a difference in the 
time it took students to complete the posttest, we ran a t-test. No significant difference was found for the time it 
took students to complete the posttest between the students working individually and students working 
collaboratively, £(74) = -1.16, p = .25. 

Second, we analyzed data from students who worked with the procedurally-oriented tutors. We tested 
whether there were statistically significant pre/post learning gains and we compared the learning gains between 
students working individually and collaboratively. We did so separately (with separate HLMs) for gains on the 
conceptual test items and gains on procedural test items. For the conceptual test items, a significant effect 







between pre and posttests was found with posttest having a higher score, £(68) = -4.49, p < .001, r = .48, while 
no significant effect for individual/collaborative learning, £(64) = 0.68, p = .5, r = .08, nor interaction, £(68) = 
-0.56, p = .58, r = .07, was found (see Figure 4). For the procedural test items, a significant effect between pre 
and posttests was found with posttest having a higher score, £(68) = -8.40, p < .001, r = .71, while no significant 
effect for individual/collaborative learning, £(64) = 0.04, p = .97, r = .005, nor interaction, £(68) = -0.57, p = .57, 
r = .07, was found (see Figure 4). Similar to what we found with the conceptually-oriented tutor, there was 
learning from pretest to posttest with the procedurally-oriented tutor, on both conceptual and procedural test 
items, but there was no difference in learning between students working individually and students working 
collaboratively. Also, no significant difference was found for the time it took students to complete the posttest 
between the students working individually and students working collaboratively, £(68) = -0.49, p = .62. 
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Figure 4 . Learning gains on the procedurally-oriented tutors. Conceptual test items on the left and procedural 
test items on the right. Collaborative students are black and individual students are light gray. 

Time on ITS 

Table 1: Mean time and problems completed on the ITS for each condition and standard deviations 


Tutor Materials 

Social Mode 

Time in Minutes: M (SD) 

Problems Completed: M (SD) 

Conceptually-oriented 

Collaborative 

73.32 (13.24) 

44.08 (11.73) 

Individual 

84.16(18.11) 

45.24 (14.50) 

Procedurally-oriented 

Collaborative 

72.78 (13.34) 

44.80 (10.29) 

Individual 

84.11 (20.68) 

45.30 (12.65) 


For the study, ample time was given to students to work with the ITS with the expectation that all students 
would complete all of problems for the assigned units. Using two t-tests, we found no difference between 
students working individually or collaboratively in the number of problems completed for the conceptually- 
oriented tutors, £(74) = -0.38, p — .l,r = .04 or the procedurally-oriented tutors, £(68) = -0.18, p = .86, r = .02 
(see Table 1). However, because the number of problems was fixed, students could work at their own pace and 
would finish the tutors at different times. Using two t-tests, there was a significant difference between students 
working individually/collaboratively on the time spent on the tutor for the conceptually-oriented tutors, £(74) = 
-2.99, p < .01, r = .33, and the procedurally-oriented tutors, £(68) = -2.78, p < .01, r = .32, where, surprisingly, 
students working collaboratively spent less time on the ITS (see Table 1). The students working collaboratively 
were able to complete the same number of problems as the students working individually but in less time. 

Discussion and Implications 

The results of our study showed significant learning gains for students working collaboratively with both the 
procedurally-oriented and conceptually-oriented tutors. There was no difference in the learning gains of students 
working collaboratively versus students working individually. However, students working collaboratively 
completed the same number of tutor problems in less time on the tutor. These results confirm our hypothesis that 
















young learners can be successfully supported in learning through the use of a collaborative ITS and indicate that 
a collaborative ITS is a viable option to use in the classroom. 

The results did not support our hypothesis that students working on conceptually-oriented tasks would 
benefit more from collaboration and students working on procedurally-oriented tasks would benefit more from 
working individually. These results are consistent with the findings from our previous work where we also 
found no difference in learning gains (Olsen et al., 2014a). However, learning with an ITS individually has been 
shown to be very successful, especially within mathematics (Ritter et al., 2007; Rau et al., 2012), and it may be 
very difficult to design an intervention that can be added to an ITS to increase learning above what can be 
achieved working individually. Collaboration adds an extra layer of complexity that might be expected to inhibit 
the learning process, even within an ITS. Yet both in our previous work and in the current study, we found 
evidence that collaborative learning with an ITS can be more efficient than learning individually with an ITS. In 
our prior study, students had the same learning gains when working collaboratively as when working 
individually, but they had practice with fewer problems (Olsen et al., 2014a). In our current study, all students 
solved the same number of problems (i.e. had the same amount of practice), but the students working in the 
collaborative condition spent less time on the tutor. This was surprising; we expected to find the opposite (Lou 
et al., 2001). Collaboration may increase the time spent on each problem if students are discussing the solution. 
This is time that would not be spent in the individual condition. Our findings that students collaborating spent 
less time on the tutor could be due to students making fewer errors when collaborating, so they spent less time 
fixing those errors. To investigate this question, future work would need to analyze the learning that happened 
within the tutor and how it may have changed over time. By analyzing the process data, we could better 
understand what actions the students were taking with the tutor and where the efficiency gains were made. 

While the collaborating students learned as much as their classmates who worked individually with 
regard to domain knowledge, they may have had more of an opportunity to develop their math reasoning skills 
and social skills by working with a partner. In a collaborative setting, students need to be able to construct their 
arguments well enough for their partner to understand their reasoning and are given the opportunity to ask 
questions (Chi & Wylie, 2014). This provides them with the opportunity to develop their math reasoning and to 
critique the reasoning of others. A next step would be to analyze the process data from the collaborative 
conditions to assess what opportunities the students had to explain their mathematical reasoning and to 
understand what the collaboration processes looked like that led to learning. A limitation of this study is that we 
did not assess social skills or mathematical reasoning skills for the students, which is where we would expect 
students collaborating to benefit beyond those working individually, as Rummel and Spada (2005) found that 
students who had an opportunity to collaborate had better knowledge about collaboration skills. 

In summary, our findings show that both collaborative and individual learning with an ITS can be 
beneficial to young students. However, the process that the students go through for the learning may be different 
when working collaboratively compared to individually as indicated by less time being spent working with the 
tutor when working collaboratively. Students may benefit from having the opportunity to work both individually 
and collaboratively in a domain. Future work is needed to investigate how to best combine individual and 
collaborative learning and where in the process each would be appropriate. Individual and collaborative learning 
may lead to different learning processes, which raises the intriguing possibility that a combination of these 
modes might support more robust learning. 

References 

Baghaei, N., Mitrovic, A., & Irwin, W. (2007). Supporting collaborative learning and problem-solving in a 
constraint-based CSCL environment for UML class diagrams. International Journal of Computer- 
Supported Collaborative Learning, 2(2-3), 159-190. doi: 10.1007/s 11412-007-9018-0 
Chen, W., & Looi, C. K. (2013). Group scribbles-supported collaborative learning in a primary grade 5 science 
class. In D. D. Suthers, K. Lund, C P. Rose, C. Teplovs, & N. Law (Eds.), Productive Multivocality in 
the Analysis of Group Interactions (pp. 257-263). Springer US. 

Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning 
outcomes. Educational Psychologist, 49(4), 219-243. 

Common Core State Standards Initiative (2015). Standards for Mathematical Practice: Construct viable 
arguments and critique the reasoning of others. Retrieved November 10, 2015 from 
http://www.corestandards.org/Math/Practice/MP3/ 

Cress, U. (2008). The need for considering multilevel analysis in CSCL research-An appeal for the use of more 
advanced statistical methods. International Journal of Computer-Supported Collaborative Learning, 
3(1), 69-84. 



Diziol, D., Walker, E., Rummel, N., & Koedinger, K. R. (2010). Using intelligent tutor technology to implement 
adaptive support for student collaboration. Educational Psychology Review, 22(1), 89-102. 

Hausmann, R. G., Chi, M. T., & Roy, M. (2004). Learning from collaborative problem solving: An analysis of 
three hypothesized mechanisms. In K.D. Rorbus, D. Gentner, & T. Regier (Eds.), Proceedings for 26nd 
Annual Conference of the Cognitive Science society (pp. 547-552). Mahwah, NJ: Erlbaum. 

Kollar, I., Fischer, F., & Hesse, F. W. (2006). Collaboration scripts-a conceptual analysis. Educational 
Psychology Review, 18(2), 159-185. 

Lazakidou, G., & Retalis, S. (2010). Using computer supported collaborative learning strategies for helping 
students acquire self-regulated problem-solving skills in mathematics. Computers & Education, 54(1), 
3-13. 

Lou, Y., Abrami, P. C., & d’Apollonia, S. (2001). Small group and individual learning with technology: A meta¬ 
analysis. Review of Educational Research, 77(3), 449-521. 

Mercer, N., & Sams, C. (2006). Teaching children how to use language to solve maths problems. Language and 
Education, 20(6), 507-528. 

Moss, J. (2005). Pipes, Tubes, and Beakers: New approaches to teaching the rational-number system. In J. 
Brantsford & S. Donovan (Eds.), How people learn: A targeted report for teachers (pp. 309-349). 
Washington DC: National Academy Press. 

Mullins, D., Rummel, N., & Spada, H. (2011). Are two heads always better than one? Differential effects of 
collaboration on students’ computer-supported learning in mathematics. International Journal of 
Computer-Supported Collaborative Learning, 6(3), 421-443. 

Murray, T. (2003). An overview of intelligent tutoring system authoring tools: Updated analysis of the state of 
the art. In T. Murray, S. B. Blessing, & S. Ainsworth (Eds.), Authoring tools for advanced technology 
learning environments (pp. 491-544). Springer Netherlands. 

Olsen, J. K., Belenky, D. M., Aleven, V., & Rummel, N. (2014a). Using an intelligent tutoring system to 
support collaborative as well as individual learning. In S. Trausan-Matu, K. E. Boyer, M. Crosby, & K. 
Panourgia (Eds.), Proceedings for 12th International Conference on Intelligent Tutoring Systems (pp. 
134-144). Berlin: Springer. 

Olsen, J. K., Belenky, D. M., Aleven, V., Rummel, N., Sewall, J., & Ringenberg, M. (2014b). Authoring tools 
for collaborative intelligent tutoring system environments. In S. Trausan-Matu, K. E. Boyer, M. 
Crosby, & K. Panourgia (Eds.), Proceedings for 12th International Conference on Intelligent Tutoring 
Systems (pp. 523-528). Berlin: Springer. 

Rau, M. A., Aleven, V., Rummel, N., & Rohrbach, S. (2012). Sense making alone doesn’t do it: Fluency matters 
too! ITS Support for robust learning with multiple representations. In S. A. Cerri, W. J. Clancey, G. 
Papadourakis, & K. Panourgia (Eds.), Proceedings for 11th International Conference on Intelligent 
Tutoring Systems (pp. 174-184). Berlin: Springer. 

Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and 
procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346- 
362. 

Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive Tutor: Applied research in 
mathematics education. Psychonomic Bulletin & Review, 14(2), 249-255. 

Rummel, N., & Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative 
problem solving in computer-mediated settings. The Journal of the Learning Sciences, 14(2), 201-241. 

Slavin, R. E. (1989). Cooperative learning and student achievement: Six theoretical perspectives. Advances in 
Motivation and Achievement, 6, 161-177. Greenwich, CT: JAI Press, Inc. 

Teasley, S. D. (1995). The role of talk in children’s peer collaborations. Developmental Psychology, 31(2), 207- 
220 . 

Tsuei, M. (2011). Development of a peer-assisted learning strategy in computer-supported collaborative learning 
environments for elementary school students. British Journal of Educational Technology, 42(2), 214- 
232. 

Walker, E., Rummel, N., & Koedinger, K. R. (2009). CTRL: A research framework for providing adaptive 
collaborative learning support. User Modeling and User-Adapted Interaction, 19(5), 387-431. 

VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other 
tutoring systems. Educational Psychologist, 46(4), 197-221. 

Acknowledgments 

We thank Amos Glenn, Ran Liu, and the CTAT team for their help. This work was supported by Graduate 

Training Grant # R305B090023 and by Award # R305A120734 from the US Department of Education (IES). 



